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ABSTRACT 

Research in education has not been relevant because: 
1) Decision makers have been making decisions without research 
findings and see no need for this kind of information now; 2) 

Research findings in the local schools are usually made available to 
the decision-maker or administrator after the decisions have been 
implemented; 3) Researchers feel the need to replicate findings 
before making them known. Several suggestions are made for making 
research findings available in a form and at times when they can be 
used by the local decision-maker. Dissemination of information which 
requires change in curriculum, administration, or methods will be 
resisted by those in power. Models which will provide information of 
process as well as product evaluation data are suggested as 
alternatives which will provide data more quickly, and provide 
partial replication before decisions are made. (Author/AG) 
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Let's Make Research More Relevant 

Some of the reasons that research in education has not 
been relevant are: 1. Decision makers have been making 

decisions without research findings and see no need for this 
kind of information now; 2. Research findings in the local 
schools are usually made available to the decision-maker 
or administrator after the decisions have been implemented; 

3. Researchers feel the need to replicate findings before 
making them known. 

Several suggestions are made for making research findings 
available in a form and at times when they can be used by 
the local decision-maker. First, the technical reports must 
be translated into simpler language which can be easily and 
quickly read by busy decision-makers. Second, testing dates 
may have to be changed so that test results will be available 
for analysis before the decisions are made; Third, the re- 
searcher will have to become more knowledgeable about the 
politics of new knowledge. 

Dissemination of information which requires change in cur- 
riculum, administration, or methods will be resisted by those 
in power. Models which will provide information of process as 
well as product evaluation data are suggested as alternatives 
which will provide data more quickly, and provide partial 
replication before decisions are made. 
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Let's Make Research More Relevant 



Wm. C. Theimer, Jr. 
University of the Pacific 



The discrepancy between research findings and decisions 
made by local school boards and school administrators makes 
most of the educational research being done meaningless in 
terms of implementation by local school districts. That 
this problem is true at other ievels of government is indi- 
cated by the report made by Dr. Dershimer in the April 1971 
Educational Researcher . He refers to "practitioners at 
the federal, state, and local levels who are key in shaping 
or reshaping major educational policies", and says: 

At present these policy influencers seldom 
consult researchers to determine how the 
findings of key studies could influence 
policies. In turn, it has also become evi- 
dent that researchers much listen more close- 
ly to the problems and concerns of the 
practitioners in order to better under- 
stand what needs to be studied to improve 
the system. 

In this paper, I will concentrate on the latter issue; 
that is, as researchers, we must learn to listen and be 
responsive to the concerns and problems of the decision- 
makers , and use our understanding of the decision-making 
process better in order to really influence the decisions 
made in educational circles today. I would like to pose 
r three factors which contribute to the problem: 

1. Decision makers have been making decisions without 
research findings for so long that they see no 
need for using this kind of information now. After 
all, the schools have been opening each fall and 
operating each year without such information, and 
everyone knows that he is an "expert" when it comes 
to educational matters anyway. 
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2. Research findings are generally made available to 
the decision-maker or administrator long after 
the decisions which required this information have 
been implemented. 

3 . Researchers are prone to want to corroborate their 
findings thoroughly before they publish the a or 
make them known to decision makers, a most desireable 
attribute but one which needs to be explored. 

It has been a practice of researchers in the past to 
blame the decision maker for not using the "good" results 
of evaluation in making decisions. As Dershimer indicated, 
this blaming the administrator indiscriminately may be 
more of a problem which the researcher should address than 
the administrator. This poses some problems in terms of 
the kind of training which has been provided most researchers 
in education. First, the researcher has been trained in 
a highly technical area in most instances (Yamamoto, 1968), 
The Title IV trainees , for example, have been given ex- 
tensive training in the use of computers , advanced statis- 
tical techniques, and experimental design, but very little 
training in the politics of disseminating the information 
they obtain. Another problem involved in the Title IV 
training program is that most of these graduates are found 
in teaching positions rather than working in research offices 

v*-n 

so that the pattern of their training will be repeated in 
the students whom they teach. 

Second, while it is true that research in education 
has been shorted in terms of money allocated for it (Wynne, 
1970 ) , it is also true • that researchers have done little to 
provide answers to questions asked by the administrators 
q at decision making time. One reason for this is that the 
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testing programs upon which most of the summative evaluation 
of new programs in the schools is based are given in May of 
each year, and by the time the results have been compiled, 
reports written, and the information disseminated, the decisions 
have long since been made regarding the planning for the new 
school year. This planning for a new school year is usually 
scheduled for February or March in order that the necessary 
materials, personnel, and other administrative details for 
beginning a new school year can be obtained and distributed. 

This timing is essential for the local school district to 
operate, and only if research results are available at 
decision making time will they be utilized. 

Third, many researchers see the production of summative 
data by March as an impossibility since the test data upon 
which much research information depends is not collected 
until May. This time schedule in most school districts 
throughout the nation is almost inflexible. But must we 
always test in May? Why not change the testing dates to 
January? Granted, the first year or so that the new testing 
program was installed, some extrapolation or interpretation 
of test data would have to be made. After a year, however, 
this could be used as a baseline, and there would be no 
reason for waiting until the end of the year to do the 
comprehensive testing. In addition if the testing were done 
at mid-year, the results could be made available to the 
teacher so that she could alter her teaching program if 
certain areas of weakness were shown to exist where he had 
^ assumed her children were proficient. This procedure would 
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also permit the data for analyzing the research programs in 
the district, which might bring about change in curriculum 
or program, to be available at the time when decisions regard- 
ing funding of such programs were being considered. 

Cohen (1970) has indicated that recent commentators, 
including Stuff lebeam and Guba, have pointed out that 
evaluation of projects in the schools is potentially rele- 
vant to decision-making. The researcher should be aware of 
the political implications of his findings, and the fact 
that the politics of a situation may make his findings of 
lesser importance than some other factors in implementing 
programs. Those trained to work in local school districts 
or state or federal agencies must be aware that many of 
the decisions which are made, not only in regard to program 
implementation and resource allocation, but also in the 
evaluation itself, are judgmental in nature (Stake, 1970). 

In this sense the discrepancy model for evaluating 
on-going programs has much merit (Provus , 1970). The re- 
searcher or evaluator recognizes that he provides information 
which may or may not be influential in making the final 
decision. Since it is a vital part of the discrepancy 
model of evaluation, the use of process data to both 
evaluate programs and corroborate previous findings should 
be emphasized. That is, a program may be instituted in a 
school system, carried out for a year and a report based on 
summative data presented to the Board of Education or 
Administration in August or September. By this time, the 
O report will have little or no effect on the program for that 
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school year, but a careful monitoring of the program during 
the fall of that year, using process data which can be 
compared to that collected the previous year could be used 
as corraborating evidence that the program should be con- 
tinued or discontinued. If the evidence is substantial 
that the program is progressing as well as it had the pre- 
vious year, and if the previous year's results indicated 
that the program was effective, then the recommendation 
would be made to continue the program. If the data had 
shown the program to be largely ineffective in accomplishing 
its goals the previous year, and the process data indicated 
that the same kinds of results could be expected during 
the coming year, this data could be used to recommend that 
the program be dropped or at least modified in a significant 
way the following year. There are, of course, many other 
possibilities for recommendations, but these should suffice 
for our purposes. 

Once the decision makers become habituated to receiving 
this kind of data, they will request that new programs 
have evaluation designs built into them, and pilot studies 
can be conducted which will provide information based on 
small samples and short periods of time to give some indi- 
cation of the usefulness of the projects. Here, less 
rigorous probability levels would probably be used, and much 
more judgmental kinds of inferences made, subject to more 
rigorous follow-up studies. 

It is surprising to many people to learn that the 
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voluminous reports which they write are actually read by 
people at the state and national levels. As I was reviewing 
one report submitted to the Bureau of Professional Develop- 
ment this past year, I noted a discrepancy and called the 
local researcher. His surprised response was, "I never 
thought that anyone ever read them so I didn 1 t check them 
carefully". This kind of .attitude has developed in many 
evaluators since much of the information required by state 
and federal agencies in particular has not provided feedback 
to them in terms of their programs’ effectiveness. In many 
cases the. kind of evaluation requested has been purely of a 
descriptive nature , and could not be used to determine 
program effectiveness in any case. As we hear more and 
more about accountability, the research reports generated 
by local, state, .and federal agencies charged with evaluative 
responsibilities will be in greater and greater demand, how- 
ever, and the purely descriptive kinds of reports will be 
replaced by more thorough reporting using the best kinds of 
evaluative techniques possible. 

A factor which will affect the degree of sophis- 
tication and care with which projects are carried out and 
evaluated is the monitoring function. By a carefully scheduled 
monitoring of projects, administrators and project directors' 
will be made more and more aware of the importance of per- 
forming the kinds of duties which have been prescribed in 
the proposals which they have submitted, or performing the 
duties required cf a particular kind of curricular innovation. 
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This monitoring process until recent years has not been 
an integral part of most locally conducted evaluation processes 
and much of the data collected has not reflected truly the 
kinds of programs for which data was collected. In other 
words, the program which was described initially was never 
really implemented, but data were collected after a prescribed 
period of time which purportedly measured that program. Such 
loose evaluations have created many of the discrepancies in 
findings which are reported in educational research literature. 

Another facet of this problem is that frequently the 
research office is called in to evaluate a program after it 
has been completed (Theimer , 1970). To stop this kind of 
request from coming from administrators who are frequently 
trying to simply get data to support their own position 
or current practice, researchers will have to build the habit 
of thinking in terms of research in the minds of the decision 
makers at all levels (Powers, 1970). In order to accomplish 
this goal, researchers will have to learn the skills needed 
to communicate with those for whom they are gathering their 
information. At present, this poses no small problems 
(Robertson, 1970, Cohen, 1971) . The fact that most of 
us feel that we must write very technical reports in order 
to impress our peers works to our discredit when we try to 
relay our findings to laymen , school boards or other 
educators. Our expertness has set us apart, and made us 
less able to communicate in simple language with this group 
of people. 
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The problems which beset us as educational researchers 
are those which must be addressed by us. Ed Wynne (1971) feels 
very pessimistic about ever effecting change through school 
administrators, and suggests that we need to be addressing 
a new constituency. Provus (197C) has devised a model 
which does work in a local school district. I think we must 
contiyuG to work with both groups based on my experiences in 
Denver and Philadelphia. 

The research department of the Philadelphia Public 
Schools, largely ignored in any decision making process for 
years, has become an increasingly important group involved 
in the planning for change in the schools. This change 
came about because (1) the department originally undertook 
to do post hoc studies, although they realized the limitations 
of them. Each time they were asked to perform one of these 
activities, however, they carefully explained that they 
should be involved in the planning process from the beginning, 
and that, their findings were not as good as they could be 
if they were involved at the beginning of a project. The 
message finally got through, and within three years after 
the reorganization of the Office of Research and Evaluation, 
superintendents were asking for reports on special projects 
as they planned their budgets for the succeeding year; (2) 
the reports were presented in simple language, with back-up 
reports of a technical nature available to support any state- 
ment made in the simplified reports; and (3) careful attention 
was paid to both the monitoring of the projects and the 
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possible to obtain in an applied situation. This change did 
not occur because the Research Department waited for the 
administration to ask them for information. It came about 
when we said that the data we had collected should be at 
least presented before decisions were made regarding re- 
allocation of funds. 

As Wynne (1970) and Cohen (1971) so eloquently point 
out, however, good information poses a real threat to 
those who are and have been working in a totally political 
realm. In that area knowledge is dangerous, and great 
skill must be employed in presenting data in such— a way that 
implementation of results can be accomplished with the 
least threat to those in power. Where the information de- 
mands that major changes in the power structure be made, 
such changes will have to be made only after concerted effort 
of a political nature. Even the changing of the time of 
testing in a school system arouses the suspicions of many 
people; to suggest major changes in organization or cur- 
riculum will certainly bring out many defensive mechanisms. 

I have suggested some of the problems which have 
made educational research irrelevant as far as implementation 
of new findings in the schools are concerned. I have also 
suggested some changes which might help to break through 
the bottleneck which keeps the schools operating several 
years behind the knowledge we now have regarding good educa- 
tional practices. To tha extent that we can begin to com- 
municate our results in understandable language and at 
appropriate times, our wotk will be effectively incorporated 
in the schools systems of our nation. 

- in 



PJRFF.REHCES 



Cohen, David K. "Politics and Research: Evaluation of Social 
Action Programs in Education" . Review of Educational 
Research , April, 1970, 

Corey, Stephen !!. ’The Local Research Program and the Research 
of Practitioners". California Journal of Educational 
Research , March, 1957. 

Dershimer , Richard A. "1971 Reoort to the Members" . Educational 
Researcher , April, 1971. 

Powers, F. F. "Research as a State of Mind". SRIS Quarterly, 
Summer, 1970. 

Provus, Malcolm. "Evaluation for Decision-Making , Choice and 
Value". Jou rnal of Research and. Development in Educa- 
tion , Summer ,’ 1*970 . " 

Robertson, Neville. "SRIS Editorial". SRIS Quarterly. Summer, 19 

Stake, Robert E. "Objectives, Priorities, and Other Judgment 
Data". Review of Educational Research, April, 1970. 



Thoimer, William C. "A .Model for Post Hoc Evaluation". AERA 
Convention, Minneapolis, Minn., 1970. 

Wall, W. D. "The Future of Educational Research". Educational 
Research , September, 1968. ” 

Walter, Robert P. ! *An Administrator Looks at Educational 

Resear ch”. Calif ornia Journal of Edu cational R esearc h , 
September, 1956. 

Wynne, Edv;ard. "Education Research: A Profession in Search 
of a Constituency; 1 ,. Phi Delta Kampan , December, 1970. 

Yamamoto , Kaoru. "A Reflection upon Research in Education”. 
The Journal of Teacher Education, Winter, 1968. 



